1. Field of the Invention
This invention relates to the field of acquiring images using an automated optical microscope system that includes a camera and particularly, to a method and system for acquiring images as devices external to the camera, which alter various parameters of the acquired images, operate asynchronously with the camera so as to allow such images to be displayed as a sequence that shows continuous variation in the acquisition parameters.
2. Discussion of the Prior Art
Performing research on living cells demands the use of an automated microscope system controlled by application software. With particular reference to the fluorescence microscopy of cells and tissues, various methods for imaging fluorescently-stained cells in a microscope and for extracting information about the spatial and temporal changes occurring in these cells are well known in the art. An article by Taylor, et al. in American Scientist 80 (1992), p. 322-335 describes many of these methods and their applications. Such methods have been particularly designed for the preparation of fluorescent reporter molecules so as to yield, after processing and analysis, spatial and temporal resolution imaging measurements of the distribution, the amount and the biochemical environment of these molecules in living cells.
As regards automating microscope systems, it is well known in the art that automated microscope systems may arrange any one of a wide variety of cameras and an array of hardware components into various instrumentation configurations, depending on the specialized research task at hand. A standard reference, especially useful for an exposition of automated optical microscopy hardware, hardware systems and system integration, is Video Microscopy, 2d Ed., 1997 by Inouxc3xa9 and Spring, which is incorporated herein by reference, especially Chapter 5. More generally descriptive of automated image acquisition and three-dimensional image visualization is John Russ""s, The Image Processing Handbook, 3d Ed., 1999, pp:1-86, 617-688 and references therein.
Also well known is that an application software package may supplement and overlay a particular instrumentation configuration by controlling and specifying the sequence, way, and functionalities of image acquiring and processing that the instrumentation system performs. The acquiring and processing operations that the software package is called upon to do depends, again, on the specialized research task at hand. The chapter titled xe2x80x9cA High-Resolution Multimode Digital Microscope Systemxe2x80x9d by Salmon et al. in Methods in Cell Biology, vol. 56, ed. by Sluder and Wolf, 1998, pp:185-215 discusses the design of a hardware system, including the microscope, camera, and Z-axis focus device of an automated optical microscope as well as application software for automating the microscope and controlling the camera.
Existing application software for automating a microscope system can direct and control a host of operations, including:
image acquisition from Recommended Standards (xe2x80x9cRSxe2x80x9d)-170 video devices, charge-coupled devices, NTSC and PAL video sources;
setting exposure time, gain, analog to digital conversion time, and bits per pixel for camera settings at each emission and/or excitation wavelength;
driving the digitizing of acquired images from an analog to digital converter;
storing acquired images in a variety of formats, such as TIFF, BMP, and other standard file formats;
driving microscope illumination;
providing capability of creating macros from a user-specified sequence of program commands, which are saved and recorded and able to be played back at a single click;
performing certain processes on a group of related images, called a stack, such as aligning images within the stack, rendering a 3-dimensional reconstruction, saving the stack to a disk, enhancing the images, deblurring the images, performing arithmetic operations; and analyzing image parameters, such as ratio imaging the concentration of ions and graphing changes in intensity and in ratios of ion concentration over time.
An example of widely-used, prior application software for automating a microscope system is the Meta Imaging Series(trademark) available from Universal Imaging Corporation, West Chester, Pa., which is a constellation of related application programs, each having a different purpose. For example, a user wanting a general, multipurpose image acquisition and processing application would employ the MetaMorph(trademark) application program; while a user needing to perform ratiometric analysis of intracellular ion measurements would employ MetaFluor(trademark).
Notwithstanding the above list of operations that prior application software can direct an automated microscope system to do, prior application software has not heretofore enabled an automated microscope system to acquire a group of images while and as acquisition parameters, such as the focus position and the emission and/or excitation wavelength, vary so that the acquired group of images can be played back as a sequence that shows continuous change in those parameters. That is, prior application software has not been capable of directing external devices that control image acquisition parameters to operate asychronously with the camera in order to acquire a group of images that may displayed as sequence showing continuous change in system parameters.
Acquiring a group of images asynchronously as a biological event is occurring so that the images can be played back as a sequence displaying continuous change in certain parameters of the microscope system has enormous importance in research with living cells. The importance of the present invention to cell research may be analogized to the importance of time-lapse photography to the study of macroscopic living systems. However, to be clear, the present invention is not merely a method akin to time-lapse photography of images acquired of living structures and processes at the cellular level. Using prior application software for processing images of cellular structures and mechanisms, a researcher is unable to observe a continuous stream of images that show uninterrupted change in system parameters other than time. The present invention allows a researcher to vary parameters, such as the position of the lens objective and the emission and/or excitation wavelength, during image acquisition so that on playback the acquired set of images may display this variability as continuous change. Specific examples of the kind of research that benefits from using the current invention include observing the movement of adhered proteins on the cell surface during live T-cell to B-cell cell-(immune cells) interactions and verifying a software model of diffusion of chemicals introduced into cells.
The following technical problem has remained unresolved by prior application software for automating an optical microscope system: namely, how to acquire images using a camera in an optical microscope system, operating at close to its maximum rate of acquisition, at the same time that external devices to the camera are continuously changing the settings of various parameters of image acquisition. The present invention solves this technical problem by providing a computerized method whereby the camera and the external devices in an automated optical microscope system are instructed to operate asychronously, that is, independently of each other, during image acquisition, thereby enabling the camera to acquire images that may be displayed as a sequence showing continuous change in image acquisition parameters.
The present invention provides a method, a computer readable medium and an automated optical microscope system for acquiring images at substantially the maximum acquisition rate of a camera while and as devices external to the camera change acquisition parameters. The stack of images so acquired images may be displayed as a sequence of images showing continuous change in the image acquisition parameters. Because the present invention directs external devices to change image acquisition parameters while and as a camera is acquiring each frame in a set of frame images, instead of directing the devices to wait until the camera has finished acquiring that frame, the camera and the external devices operate asynchronously. In different terms, the present invention directs external devices to operate to change the image acquisition parameter they control, for example, the focus position of the microscope objective lens, the emission and/or excitation wavelength or the position of the microscope stage, while and as a camera is acquiring an image.
The method of the present invention comprises the following steps:
a) configuring an image-acquiring system comprising an automated microscope, a camera, devices external to the camera for altering the image acquisition parameters of focus plane, excitation wavelength and/or emission wavelength and a computer, whereby the external devices are directed to acquire images asychronously with the camera;
b) acquiring images at a rate substantially close to the maximum image acquisition rate of the camera and storing the acquired images as digitized data;
c) during the acquiring and storing of images, operating at least one said external device whereby at least one image acquisition parameter is altered;
The method of the present invention may be used under a wide variety of microscopy modes, including brightfield, fluorescence, darkfield, phase contrast, interference, and differential interference contrast (DIC). One of the embodiments of the method of the present invention is as a set of instructions resident in an information handling system. Another embodiment of the method of the present invention is as a set of instructions resident in a computer readable medium.
It is a feature of the present invention that a group of images, called a stack, may be acquired as the focus position changes are made so that the group of images so acquired may be displayed as a sequence showing continuous change of the Z-position. It is a further feature of the present invention that a stack of images may be acquired as changes to the emission and/or excitation wavelength are made so that a group of images may be displayed as a sequence showing continuous change of the emission and/or excitation wavelength. Further, it is a feature of the present invention that a stack of images may be acquired as changes to various acquisition parameters, such as the focus position and the emission and/or excitation wavelength are made in concert so that a group of images displayed as a sequence show continuous change in the various acquisition parameters selected. A feature of another embodiment of the present invention is that at regular time intervals, a stack of images may be acquired while and as various image parameters are simultaneously changed so that the stacks of images may be displayed as a sequence that shows continuous change over time and continuous change in acquisition parameters.
An advantage of the present invention is that a stack of images allows a three-dimensional rendering of the relevant cellular mechanism, process, and/or structure during a biological event, such as the introduction of a chemical into a cell or cell division. A further advantage is that multiple stacks of images allow a three-dimensional rendering of a biological event of interest as image acquisition parameters change and over a selected time period.